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Abstract 


The  United  States  has  increasingly  relied  on  unmanned  systems  to  help  fight  its  wars, 
and  these  systems  have  steadily  become  more  sophisticated.  Science  is  now  on  the  verge  of 
many  breakthroughs,  particularly  in  artificial  intelligence,  which  might  well  lead  to  the 
creation  of  fully  autonomous  unmanned  systems.  Autonomous  systems  could  potentially 
attack  enemies  with  lethal  force  without  a  human  being  in  the  loop.  The  possible  deployment 
of  lethal,  autonomous  robots  raises  significant  legal  and  ethical  concerns.  These  profound 
concerns,  including  whether  such  systems  would  even  comport  with  the  Law  of  Armed 
Conflict,  have  not  yet  been  definitively  resolved.  The  technology,  however,  continues  to 
race  forward,  and  many  nations  of  the  world  are  actively  pursuing  autonomous  programs.  To 
prevent  being  surpassed  by  rivals,  the  U.S.  will  likely  exploit  the  potential  of  fully 
autonomous  targeting.  Therefore,  operational  commanders  should  begin  examining  the  legal 
and  the  command  and  control  implications  of  using  such  lethal,  autonomous  robots  as  they 
help  steer  the  future  development  and  doctrine  of  unmanned  systems.  While  the  use  of  such 
systems  will  arguably  be  deemed  permissible  under  the  Law  of  Armed  Conflict,  prudent 
operational  commanders  should  still  implement  additional  control  measures  to  increase 
accountability  over  these  new  systems. 


Autonomous  robots  on  the  battlefield  will  be  the  norm  within  twenty  years. 1 

I.  Introduction/Thesis. 

Robots  and  unmanned  systems  have  proven  to  be  incredibly  valuable  on  the 
battlefield  during  the  Global  War  on  Terror  and  are  likely  to  play  a  larger  and  more 
sophisticated  role  for  militaries  in  the  future.  From  2000  through  2010,  the  number  of  U.S. 
unmanned  aerial  vehicles  (UAVs)  proliferated  from  fewer  than  fifty  to  over  seven  thousand, 
with  similarly  astounding  increases  among  land  and  sea-based  unmanned  systems.-  Despite 
overall  reductions  in  upcoming  U.S.  defense  budgets,  expenditures  for  unmanned  systems  are 
projected  to  grow.  All  branches  of  the  U.S.  military  are  poised  to  rely  more  heavily  on 
unmanned  systems  in  the  future.4  Not  only  are  the  numbers  of  these  systems  increasing  but 
so  are  their  capabilities.  Technology  has  advanced  so  remarkably  in  the  past  few  years, 
particularly  with  respect  to  artificial  intelligence,  that  the  creation  of  fully  autonomous 
systems  appears  to  be  a  distinct  possibility  in  coming  years.  The  potential  deployment  of 
fully  autonomous,  lethal  systems  raises  significant  legal  and  ethical  concerns.  These 
profound  concerns,  including  whether  such  systems  would  even  comport  with  the  Law  of 
Armed  Conflict  (LOAC),  have  not  yet  been  definitively  resolved.  The  technology,  however, 
continues  to  race  forward  regardless.  Therefore,  operational  commanders  should  begin 
examining  the  legal  and  the  command  and  control  implications  of  using  such  lethal, 
autonomous  robots  (LARs)  as  they  help  steer  the  future  development  and  doctrine  of 
unmanned  systems.5  While  the  use  of  LARs  will  arguably  be  deemed  permissible  under 
LOAC  in  most  circumstances,  prudent  operational  commanders  should  still  implement 
additional  control  measures  to  increase  accountability  over  such  systems. 
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II.  Technological  Advances  May  Make  Lethal,  Autonomous  Robots  Possible. 

Operational  commanders  need  to  be  aware  of  recent  technological  advances  and  the 
extent  to  which  the  military  is  poised  to  incorporate  these  advances  into  future  unmanned 
systems.  While  LARs  may  seem  incredibly  futuristic  at  first  blush,  the  technological  gap  is 
quickly  narrowing.  In  fact,  the  former  Chief  Scientist  for  the  U.S.  Air  Force  even  contends 
that  technology  currently  exists  to  facilitate  “fully  autonomous  military  strikes.”6  Several 
recent  technological  breakthroughs,  particularly  those  involving  artificial  intelligence, 
highlight  how  attainable  these  systems  are  becoming. 

A.  Recent  Artificial  Intelligence  Breakthroughs  Are  Creating  Vast  Opportunities. 

The  past  few  years  have  witnessed  tremendous  technological  breakthroughs  in 
artificial  intelligence.  Two  highly  publicized  examples  showcase  the  extraordinary  potential 
of  artificial  intelligence.  The  first  involves  the  IBM  supercomputer  system  known  as 
“Watson.”  The  Watson  supercomputer  is  best  known  for  competing  and  winning  against 
human  competitors  on  the  Jeopardy  television  quiz  show  during  several  special  episodes 
which  aired  in  February  2011.  The  uniqueness  of  Watson  stemmed  from  the  way  it  learned 
to  identify  the  answers  to  the  trivia  questions.  To  attempt  to  replicate  the  complex  human 
thought  process,  Watson  was  designed  with  more  than  one-hundred  statistical  algorithms. 
The  algorithms  helped  Watson  rapidly  sort  through  multiple  databases  of  stored  information. 
The  algorithms  essentially  helped  Watson  learn,  statistically  speaking,  which  words  were 
most  likely  associated  with  which  answers.  Watson  marked  a  revolutionary  advance  in 
artificial  intelligence  both  in  the  number  of  algorithms  embedded  into  it  and  in  the  statistical 
methods  it  utilized  in  problem  solving.  The  extraordinary  technology  showcased  in  Watson 
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will  likely  begin  appearing  in  other  computer  systems  and  could  be  adapted  to  assist  LARs  in 

o 

the  future.  Watson,  however,  is  but  one  recent  breakthrough  in  artificial  intelligence. 

A  second  technological  breakthrough  came  from  Google  with  its  driverless  car. 
Google  funded  a  team  of  researchers  to  design  vehicles  that  could  drive  without  human 
controllers  on  city  streets  and  public  highways.  The  researchers,  most  of  whom  are  part  of 
Stanford  University’s  Artificial  Intelligence  Laboratory,  successfully  created  seven  vehicles, 
which  navigated  California’s  freeways  and  streets  accident-free  for  approximately  140,000 
miles  with  only  sporadic  human  assistance.9  The  sophisticated  artificial  intelligence  in  these 
vehicles  was  able  to  “sense  anything  near  the  car  and  mimic  the  decisions  made  by  a  human 
driver.”10  This  cutting-edge  technology  represented  a  tremendous  leap  forward  in  artificial 
intelligence.  The  potential  military  use  of  systems  capable  of  navigating  themselves  is  clear. 
In  fact,  this  Google  project  was  an  extension  of  an  earlier  Stanford  University  project  which 
competed  in  and  won  the  2005  Defense  Advanced  Research  Projects  Agency  (DARPA) 
Grand  Challenge  competition.  That  Pentagon-funded  competition  offered  a  two-million 
dollar  prize  to  the  team  who  could  develop  an  autonomous  vehicle  capable  of  navigating 
itself  over  a  one-hundred  thirty  mile  desert  course.1 1  The  Google  version  of  the  vehicle 
represents  a  marked  improvement  over  the  one  which  won  the  DARPA  prize,  and  possesses 
the  advanced  artificial  intelligence  capabilities  that  the  military  will  likely  incorporate  into 
future  unmanned  systems. 

The  true  breakthrough  of  systems  like  Watson  and  the  Google  car  is  the  way  in  which 
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they  adapt  and  learn.  These  systems  essentially  are  able  to  leam  from  their  own  mistakes. 
The  artificial  intelligence  branch  used  in  these  systems  is  called  “machine  learning.”  The 
computers  can  recognize  patterns  in  data  and  leam  from  the  patterns  to  make  decisions  or 
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perform  a  function.14  It  is  akin  to  humans  learning  through  examples.15  Machine  learning  is 
helping  computer  developers  tackle  problems  “once  thought  too  complex  for  computers.”16 

Any  future  development  of  LARs  will  rely  heavily  on  such  types  of  artificial 
intelligence  reasoning  capabilities.  Machine  learning  computers  will  likely  help  future  LARs 
learn  the  necessary  behaviors  and  make  the  critical  decisions  about  whether  and  how  to 
engage  and  destroy  a  target.  The  U.S.  military  has  wisely  positioned  itself  to  incorporate 
these  new  technological  breakthroughs  into  the  next  generation  of  its  unmanned  systems. 

B.  U.S.  Military  is  Poised  to  Capitalize  on  Technological  Breakthroughs. 

The  Department  of  Defense  (DOD)  is  at  the  vanguard  of  developing  new  unmanned 
technologies.  DARPA  is  the  “primary  player  in  the  world  of  funding  new  research  in  . . . 
robotics.”  7  It  sponsors  research  on  future  technologies,  and  is  currently  focused  heavily  on 
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robots  and  unmanned  systems.  Other  government  entities,  like  the  Office  of  Naval 
Research  (ONR),  are  funding  efforts  to  develop  robots  that  can  act  independent  of  humans.19 
These  DOD  organizations  helped  create  the  vast  numbers  of  unmanned  systems  that 
deployed  to  Afghanistan  and  Iraq  over  the  past  decade  of  fighting.20  They  are  now  poised  to 
develop  even  more  sophisticated  systems  in  coming  years. 

As  the  technology  advances,  many  newer  DOD  unmanned  systems  are  taking  greater 
advantage  of  these  artificial  intelligence  improvements  and  are  being  designed  with  more 
autonomous  features.  For  instance  in  the  U.S.  Navy,  close-in  weapons  systems,  like  the 
Phalanx  found  on  Aegis-class  cruisers  and  other  ships,  can  now  autonomously  find,  track, 
and  destroy  enemy  anti-ship  missiles.  ONR  is  developing  systems  for  the  U.S.  Navy  like 
the  Biomimetic  Autonomous  Undersea  Vehicle  (BAUV),  which  is  capable  of  conducting 
long-term  underwater  surveillance.  BAUV  can  recognize  changes  in  the  environment  and 
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make  adjustments  autonomously  to  retain  its  position  in  the  water  for  many  weeks. ""  The 
U.S.  Navy  is  also  developing  “mine-hunting,”  autonomous  mini-submarines. 

The  U.S.  Navy  is  not  alone  in  pursuing  unmanned  systems  with  autonomous  features. 
The  U.S.  Air  Force  has  designed  its  Global  Hawk  UAV  systems  to  include  autonomous 
flying  options."  Rather  than  directly  controlling  the  aircraft’s  every  move,  human  operators 
merely  designate  the  areas  the  Global  Hawk  is  to  observe.  The  system  then  flies  itself  to 
those  areas  utilizing  GPS  satellites."  The  U.S.  Air  Force  is  also  researching  the  use  of 
Proliferated  Autonomous  Weapons  or  PRAWNs,  which  are  systems  of  small  robots  that 
could  potentially  be  flown  autonomously  to  attack  targets  as  a  swarm.26 

The  U.S.  Army  has  been  developing  a  series  of  unmanned  vehicles  capable  of 
autonomous  operations.  Some  future  U.S.  Army  counter  battery  systems  may  be  able  to 
autonomously  destroy  incoming  artillery  and  missile  barrages  at  speeds  faster  than  humans 
could  possibly  perform.  Other  U.S.  Army  unmanned  ground  systems  are  being  designed  to 
move  around  the  battlefield  autonomously,  such  as  the  Crusher  Unmanned  Ground  Combat 
Vehicle.  The  Crusher  possesses  advanced  artificial  intelligence  capabilities  and  represents  a 
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potential  prototype  of  the  next  generation  autonomous  robotic  ground  fighting  vehicle. 

In  anticipation  of  these  autonomous  features  becoming  more  widely  available,  DOD 
is  already  developing  doctrine  and  tactics  for  incorporating  autonomous  systems  into  the 
overall  force.  Military  organizations,  such  as  DARPA,  ONR,  and  the  U.S.  Army  Research 
Laboratory  have  been  working  diligently  on  the  so-called  “warfighters’  associate”  concept, 
which  will  partner  humans  and  robots  together  to  work  as  “synergistic  teams.”"  The 
expectation  is  that  robots  on  the  battlefield  in  the  future  will  form  the  bulk  of  detachments, 
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such  as  infantry  units  that  would  be  comprised  of  150  human  soldiers  working  alongside 
2,000  robots.30 

Operational  commanders  need  to  be  aware  not  only  that  these  technological 
breakthroughs  will  make  autonomous  features  more  readily  available  but  also  that  there 
likely  will  be  a  growing  need  for  unmanned  systems  to  become  more  autonomous  in  the 
future.  Several  key  reasons  explain  the  growing  need.  First,  requiring  a  man-in-the-loop  for 
all  unmanned  systems  is  prohibitive  personnel-wise.  It  takes  scores  of  people,  from  pilots  to 
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technicians  to  intelligence  analysts,  to  operate  a  single  tethered  UAV.  Particularly  as  the 
overall  size  of  the  force  shrinks  in  coming  years,  more  sophisticated  unmanned  systems  will 
be  expected  to  fill  the  capability  gaps.  Second,  future  battles  will  likely  occur  at  such  a  fast 
tempo  that  human  controllers  may  not  be  able  to  supervise  drone  forces  quickly  enough  to 
counter  what  the  enemy  may  be  doing.  Essentially,  a  force  in  the  future  that  does  not  have 
fully  autonomous  systems  may  not  be  able  to  compete  with  an  enemy  who  does.  Many 
nations  around  the  world,  including  China,  are  already  developing  advanced  systems  with 
autonomous  features.34  Third,  adversaries  are  improving  satellite  communications  jamming 
and  cyber-attack  capabilities,  and,  as  a  result,  systems  tethered  to  a  human  controller  may  be 
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incredibly  vulnerable.  Without  a  constant  connection  to  a  human  operator,  tethered 
systems  are  incapable  of  completing  their  missions.36  Thus,  in  general,  future  weapons 
systems  will  be  “too  fast,  too  small,  too  numerous,  and  will  create  an  environment  too 
complex  for  humans  to  direct.”  One  likely  solution  will  be  unmanned  systems  that  are 
much  more  autonomous  than  those  that  presently  exist. 

Although  the  U.S.  is  developing  a  variety  of  autonomous  features  for  many  of  its 
unmanned  systems  as  discussed  above,  the  U.S.  remains  committed,  at  the  moment,  to 
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having  a  human  remain  in  the  loop  for  most  lethal  targeting  decisions.  One  of  the  main 
reasons  the  U.S.  has  not  yet  fully  embraced  lethal,  autonomous  targeting  is  the  legal 

on 

uncertainty  associated  with  robots  making  those  life  and  death  decisions.  As  will  be 
explored  fully  below,  deciding  whether  LARs  are  permissible  under  LOAC  remains  an 
unresolved  and  hotly  contested  issue. 

III.  LOAC  Would  Permit  Fully  Autonomous  Targeting  Under  Most  Circumstances. 

LOAC  has  proven  to  be  flexible  and  has  evolved  and  adapted  over  time  to  advances 
in  both  weapons  technology  and  military  tactics.40  Many  weapons  systems  were  initially 
outlawed  only  to  be  accommodated  later,  once  the  technology  proliferated  to  other  nations 
and  international  norms  conformed.41  LOAC  is  essentially  derived  from  customary 
international  practices  and  international  treaties,  but  thus  far  there  is  neither  international 
consensus  nor  an  international  treaty  about  autonomous  targeting.  Internationally,  the 
debate  over  whether  LARs  should  be  lawful  is  highly  contentious.43  Any  examination  of  the 
lawfulness  of  LARs  must  begin  with  the  aspect  of  LOAC  known  as  jus  in  bello  (or  justice  in 
war),  which  focuses  on  determining  the  practices  allowed  and  the  practices  prohibited  in 
war.44  The  jus  in  bello  is  comprised  of  four  bedrock  principles:  military  necessity, 
distinction,  proportionality,  and  unnecessary  suffering.45  As  will  be  discussed  below,  with  a 
careful  analysis  of  these  and  other  foundational  LOAC  principles,  the  use  of  LARs  will  likely 
be  deemed  permissible  in  the  vast  majority  of  circumstances. 

A.  Military  Necessity. 

LOAC  is  not  designed  to  hinder  the  waging  of  war  but  is  instead  intended  to  ensure 
combatants  properly  direct  violence  toward  the  “enemy’s  war  efforts.”46  The  principle  of 
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military  necessity  helps  to  achieve  that  goal.  Military  necessity  requires  combatants  to  focus 
their  military  efforts  and  attacks  on  those  items  with  a  military  objective  or  those  offering  a 
“definite  military  advantage.”47  Thus,  force  may  only  be  used  when  it  will  help  the 
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belligerent  win  the  war.  Belligerents  are  expected  to  examine  whether  an  “object  of  attack 
is  a  valid  military  objective”  before  engaging  a  particular  target.49  One  normally  looks  to  an 
object’s  nature,  location,  use,  or  purpose  to  determine  if  it  is  indeed  a  valid  military  target. 

Given  those  parameters,  LARs  would  need  to  be  able  to  make  the  determination  that 
a  potential  target  meets  the  criteria  as  a  valid  military  objective.  While  this  decision  making 
process  might  be  complex,  forces  utilizing  the  unmanned  systems  would  be  able  to  greatly 
influence  this  process  and  likely  ensure  compliance  with  the  LOAC  principle.  Even  though  a 
system  is  designed  to  operate  autonomously,  it  would  presumably  be  given  specific  orders 
from  its  headquarters  about  what  types  of  missions  it  would  be  expected  to  accomplish. 
Leadership  would  most  likely  program  LARs  to  only  engage  specific  targets  or  at  least 
specific  types  of  targets.  In  essence,  the  systems  would  be  told  who  its  enemy  is  and  what 
objects  belong  to  that  enemy.  As  long  as  the  types  of  targets  and  missions  assigned  to  LARs 
are  valid  military  objectives,  then  the  LARs  would  be  in  compliance  with  the  principle  of 
necessity  when  engaging  those  targets. 

The  issue  becomes  more  complicated  if  the  target  is  not  on  a  preset  list  of  targets. 
Such  a  situation  might  arise  when  a  target  is  a  “target  of  opportunity”  or  is  in  response  to 
some  kind  of  emergency  situation.  The  most  likely  emergency  situation  is  one  in  which 
fellow  troops  are  being  attacked  and  LARs  are  dispatched  to  provide  the  troops  assistance. 

In  those  situations,  the  military  necessity  prong  would  be  relatively  easy  to  meet  as  part  of  a 
unit  self-defense  argument.  Nevertheless,  as  will  be  discussed  in  Section  IV  below, 
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operational  commanders  may  still  want  to  limit  LARs  from  engaging  targets  in  some  such 
emergency  situations. 

B.  Distinction. 

The  jus  in  bello  principle  of  distinction  requires  belligerents  to  distinguish  between 
combatants  and  civilians.50  It  applies  to  both  real  persons  and  tangible  objects.51  The  intent 
is  to  minimize  the  harm  to  civilians  and  their  property.  Commanders  have  the  affirmative 
duty  to  distinguish  between  these  before  ordering  an  attack.  This  principle  is  intended  to 
prohibit  indiscriminate  attacks. 

LARs  have  the  same  requirements  to  distinguish  as  any  other  member  of  the  force. 
They  need  to  be  able  to  identify  between  civilian  and  military  objects  and  personnel.  To 
make  this  distinction,  LARs  should  be  able  to  rely  upon  uniforms  and  other  distinctive  signs. 
Given  the  advanced  image  recognition  technology  expected  to  be  incorporated  into  LARs, 
the  systems  will  likely  be  capable  of  recognizing  this  distinction  consistently.54 

As  the  U.S.  has  learned  during  the  past  decade  of  fighting,  however,  enemies  do  not 
always  wear  uniforms  or  use  distinctive  marks.  In  such  uncertain  cases,  civilians  are 
safeguarded  “unless  and  for  such  time  as  they  take  a  direct  part  in  hostilities.”55  Determining 
if  and  when  a  civilian  is  taking  direct  part  in  hostilities  can  often  be  a  most  difficult  decision 
to  make.  Similar  to  humans,  LARs  would  have  a  difficult  time  making  this  distinction.56 
However,  LARs  possess  one  advantage  over  humans  in  this  regard.  They  are  not  constrained 
by  the  notion  of  self-preservation.  Thus,  LARs  could  be  programmed  to  sacrifice  themselves 

C-l 

to  “reveal  the  presence  of  a  combatant.”  LARs  could  easily  be  ordered  to  hold  their  fire 
until  after  being  fired  upon  by  the  enemy.  In  so  doing,  the  use  of  LARs  could  greatly  help  a 
belligerent  distinguish  combatants  from  non-combatants  on  a  complex  battlefield. 
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Belligerents  would  still  need  to  satisfy  the  other  foundational  principles,  including 
proportionality. 

C.  Proportionality. 

Proportionality  requires  belligerents  to  weigh  the  military  advantage  of  their  attack 
against  the  unavoidable  collateral  damage  that  will  result.58  An  attack  is  lawful  as  long  as  it 
is  not  expected  to  cause  collateral  damage  that  would  be  “excessive”  in  relation  to  the 
military  advantage.59  Thus,  collateral  damage  is  permitted  but  only  in  an  amount  that  would 
not  be  deemed  excessive.  It  is  vital  to  recognize  that  the  balancing  decision  is  made  in 
anticipation  of  the  attack  rather  than  with  the  actual  amount  of  collateral  damage  caused  after 
the  fact.60 

This  proportionality  determination  equates  to  a  judgment  call.  The  judgment  call  has 
always  belonged  to  a  human.  Traditionally,  the  call  has  been  compared  against  what  a 
“reasonable  person”  or  a  “reasonable  commander”  would  do  in  such  a  situation.  As  long  as  a 
similarly  situated  person  would  be  expected  to  make  a  comparable  determination  of  what  is 
excessive  under  the  circumstances,  then  the  decision  to  strike  would  be  deemed  lawful.61 
Advances  in  artificial  intelligence  notwithstanding,  it  remains  unclear  whether  a  robot’s 
determination  of  excessiveness  could  be  considered  sufficient  given  such  a  standard.62 

Even  if  the  proportionality  standard  represented  an  obstacle,  many  workarounds 
might  still  enable  commanders  to  lawfully  employ  LARs  on  the  battlefield.  Operational 
commanders  could  use  LARs  in  situations  where  a  higher  amount  of  collateral  damage  might 
be  acceptable.  Normally,  attacks  directed  against  high  value  targets  or  against  a  declared 
hostile  force  in  a  high-intensity  conflict  might  fall  into  this  category.63  Similarly,  a 
commander  could  designate  a  bright-line  rule  for  the  amount  of  expected  collateral  damage 


10 


that  is  permissible  during  a  specific  mission.  Thus,  if  LARs  determine  that  the  expected 
number  of  civilian  casualties  exceeds  the  predetermined  acceptable  limit,  they  would  not  be 
permitted  to  engage  the  target  without  supplementary  human  approval.  These  and  other 
additional  control  measures  will  be  more  fully  addressed  in  Section  IV  below.  Beyond 
proportionality,  the  U.S.  must  also  ensure  LARs  do  not  cause  unnecessary  suffering. 

D.  Unnecessary  Suffering. 

The  last  jus  in  bello  principle  is  unnecessary  suffering  or  humanity.  When  examining 
the  lawfulness  of  LARs,  this  principle  should  not  prevent  their  use  as  long  as  standard 
munitions  and  tactics  are  used  in  these  robots.64  LOAC  requires  belligerents  to  prevent 
unnecessary  suffering  when  conducting  attacks.  To  comply,  belligerents  cannot  use  any 
weapon  or  ammunition  that  is  calculated  to  cause  such  harm.65  Instead,  they  must  only  use 
lawfully  designed  weapons  and  ammunition  and  employ  them  in  a  lawful  method  of  warfare. 
All  U.S.  military  weapons  and  ammunition  have  been  designed  with  these  considerations  in 
mind.  As  a  result,  the  U.S.  does  not  field  per  se  unlawful  munitions,  such  as  hollow-point 
rounds  or  warheads  filled  with  glass.66  In  this  case,  LARs  equipped  with  standard  weapons 
and  ammunition  and  used  in  accordance  with  U.S.  doctrine  would  likely  be  deemed  to 
comply  with  the  principle  of  unnecessary  suffering. 

Overall,  as  explained  in  the  preceding  paragraphs,  LARs  would  arguably  be  in 
compliance  with  all  four  foundational  jus  in  bello  principles  in  the  vast  majority  of 
circumstances.67  Commanders  should,  therefore,  be  confident  in  their  ability  to  utilize  LARs, 
especially  given  some  of  the  additional  control  measures  that  will  be  discussed  in  Section  IV 
below.  This  opinion  on  the  lawfulness  of  LARs  is  by  no  means  universal,  however.  Many 
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legal  commentators  argue  that  LARs  should  be  banned  under  international  law.  Some  of  the 
key  criticisms  are  discussed  below. 

E.  Counterarguments:  Lawfulness  of  LARs  is  Disputed. 

There  are  several  strong  counterarguments  for  why  LARs  might  not  be  permissible 
under  LOAC.  First,  many  critics  argue  that  LOAC  assumes  a  human  is  ultimately  making 
the  weighty  life  and  death  decisions.  It  would,  therefore,  be  morally  wrong  to  completely 
remove  humans  from  these  targeting  decisions.  As  such,  LARs  operate  outside  the  bounds  of 
the  applicable  international  laws  and  norms.68  Second,  other  critics  contend  that  the  systems 
should  be  deemed  as  illegal  because  their  use  could  lead  to  a  total  lack  of  accountability  for 
attacks  on  civilians.  They  assert  that  there  is  no  human  who  can  be  held  accountable  for  a 
breach  committed  by  an  autonomous  system.69  Those  critics  contend  that  there  is  a  “visceral 
human  desire  to  find  an  individual  accountable.”  Third,  other  critics  argue  that  the  fact  that 
a  system  is  technologically  possible  does  not  automatically  mean  it  is  lawful.  They  contend 
that  some  weapons  systems  are  simply  too  dangerous  and  thus  risk  causing  too  much 
unnecessary  suffering.  They  argue  that  other  systems,  such  as  lasers  with  the  ability  to  blind 
soldiers  on  a  battlefield,  are  technologically  possible  but  have  been  banned  from  war  for 
being  too  abhorrent.  They  contend  that  LARs  should  suffer  a  similar  fate.  Fourth,  still 
other  critics  contend  that  LARs  fail  the  proportionality  test,  for  some  of  the  reasons  that  were 
discussed  above.  In  particular,  they  argue  that  robots  will  not  be  able  to  “holistically  weigh” 
the  proportionality  test.  “  While  LARs  may  be  able  to  determine  if  the  number  of  expected 
civilian  casualties  exceeds  some  predetermined  limit,  the  proportionality  test  requires  a 
greater  sense  of  what  is  excessive. 
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While  those  critics  provide  compelling  reasons  to  doubt  the  lawfulness  of  LARs,  their 
counterarguments  can  be  rebutted  with  a  deeper  examination  of  the  many  prevailing  theories 
on  the  law.  The  first  counterargument  questioned  whether  LOAC  is  designed  to  handle  life 
and  death  decisions  made  by  robots  vice  humans.  LOAC  is  indeed  a  flexible  and  robust 
body  of  law.  It  has  adapted  to  numerous  technological  changes,  such  as  the  development  of 
submarines  and  helicopters  and  nuclear  weapons.  Although  the  development  of  LARs 
represents  a  significant  advancement  in  war  fighting,  it  is  not  so  drastic  a  change  as  to 
warrant  throwing  out  the  existing  body  of  international  laws.  LOAC  can  evolve  to 
encompass  LARs  and  can  provide  necessary  and  sound  guidance  to  their  use.  The  second 
counterargument  focused  on  the  lack  of  accountability.  Contrary  to  the  opinions  of  those 
critics,  LOAC  does  not  require  that  a  human  be  held  personally  accountable  for  any  mistakes 
or  violations  that  may  occur  on  the  battlefield.  While  the  need  to  hold  someone  accountable 
might  be  “visceral,”  it  is  not  definitively  required  by  law.  Instead,  international  law  demands 
that  states  not  absolve  themselves  of  liability  with  respect  to  a  grave  breach  of  the  laws  of 
war.74  Therefore,  the  state  would  likely  be  responsible  for  any  breach  related  to  LARs.75 
Such  a  framework  essentially  exists  today  if,  for  instance,  a  sophisticated  mine  exploded 
incorrectly  and  injured  a  civilian  or  some  civilian  property.  The  lack  of  a  human  to  hold 
accountable  does  not  undermine  the  lawfulness  of  the  weapon  system.76 

With  respect  to  the  third  counterargument  regarding  abhorrent  weapons,  LARs  can 
easily  be  distinguished  from  blinding  lasers  and  other  banned  weapons.  As  opposed  to  those 
weapons  where  the  weapon  itself  was  at  issue,  the  unique  feature  of  LARs  is  the  autonomous 
controls.  LARs  are  expected  to  use  the  same  types  of  conventional  munitions  found  on 
manned  military  systems,  and  the  lethality  of  LARs  would  not  differ  substantially  from  that 
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of  other  weapons  systems.  Thus,  LARs  would  not  cause  the  same  type  of  unnecessary 
suffering  that  blinding  lasers  create.  As  such,  it  seems  less  likely  that  LARs  would  be 
deemed  abhorrent  under  international  law. 

The  fourth  counterargument  dealt  with  proportionality  and  the  requirement  for  a 
holistic  approach.  As  was  discussed  above,  the  proportionality  judgment  call  is  normally 
assumed  to  be  a  human’s  decision.  While  it  is  not  clear  whether  a  robot’s  judgment  will  be 
deemed  holistic  enough  for  the  critics,  the  commander’s  orders  and  guidance  to  LARs  about 
acceptable  levels  of  collateral  damage  may  be  sufficient  to  encompass  that  holistic 
examination.  Furthermore,  there  is  actually  no  specific  LOAC  requirement  for  the  judgment 
call  to  be  holistic.  International  law  merely  requires  belligerents  to  balance  the  military 
advantage  against  the  expected  collateral  damage.  Thus,  critics  are  expanding  the  notion  of 
proportionality  beyond  what  is  legally  required. 

In  general,  such  strong  counterarguments  highlight  just  how  complicated  and 
unresolved  these  legal  issues  remain.  Given  this  complexity,  prudent  operational 
commanders  will  enact  additional  control  measures  when  utilizing  LARs. 

IV.  Prudent,  Additional  Control  Measures  For  Commanders  of  LARs. 

Even  though  LARs  will  likely  be  technologically  possible  and  permitted  under  LOAC 
in  the  future,  operational  commanders  would  be  wise  to  plan  carefully  for  how  and  when  to 
utilize  such  systems.  There  may  be  situations  in  which  using  LARs  might  actually  prove 
disadvantageous  and  unnecessarily  risky.  If  an  operational  commander  ever  doubts  the 
effectiveness  or  lawfulness  of  using  LARs  in  a  particular  situation,  then  the  commander 
should  not  deploy  them  or  instead  implement  additional  control  measures  to  further  protect 


14 


the  unit  from  LOAC  violations.  The  following  additional  control  measures  will  assist 
operational  commanders  in  their  employment  of  LARs  systems. 

First,  operational  commanders  need  to  ensure  that  all  LARs  have  the  proper  rules  of 
engagement  (ROE),  tactical  directives,  and  other  national  caveats  embedded  into  their 
algorithms.  Moreover,  commanders  must  ensure  any  revisions  to  the  ROE  or  directives  are 
immediately  inputted  into  and  incorporated  by  the  LARs.  Unmanned  underwater  systems, 
particularly  those  without  regular  communications  with  the  headquarters,  may  prove  most 
challenging  in  this  arena.  For  LARs  that  cannot  make  such  adjustments  while  deployed, 
commanders  need  to  ensure  those  systems  can  be  recalled  and  then  reprogrammed  quickly  in 
the  event  of  such  a  change. 

Second,  commanders  should  limit  when  and  where  LARs  are  employed  to  avoid 
potential  proportionality  issues.  Geographically,  LARs  are  best  suited  to  engage  targets  in 
areas  where  the  likelihood  of  collateral  damage  is  reduced,  such  as  underwater  or  in  an  area 
like  the  demilitarized  zone  in  Korea.  Regardless  of  geography,  LARs  might  be  appropriate 
when  the  target  is  one  of  particularly  high  value.  In  such  situations,  a  commander  may  have 
fewer  proportionality  concerns  or  might  at  least  be  able  to  quantify  the  amount  of  acceptable 
collateral  damage.  Utilizing  LARs  only  in  “kill  box”  type  environments  or  when  pursuing 
high  value  targets  would  alleviate  many  of  the  critics’  proportionality  concerns  and  best 

70 

protect  operational  commanders. 

Third,  operational  commanders  should  carefully  examine  the  type  of  conflicts  in 
which  they  deploy  LARs.  Commanders  would  be  wise  to  use  LARs  predominantly  during 
high-intensity  conflict  situations  where  the  ROE  is  status-based,  meaning  there  is  a  declared 
hostile  force  to  attack.  Those  declared  hostile  forces  would  then  be  easily  recognizable, 
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eligible  targets  for  LARs.  LARs  are  less  appropriate  in  counterinsurgency  or  irregular 
warfare  situations,  where  “the  blurring  of  the  lines  between  civilian  and  military  is  a 
commonplace  occurrence.”  Similarly,  commanders  may  also  want  to  restrict  LARs  in 
emergency  type  situations  where  the  proposed  target  is  not  already  on  a  preset  list  of  targets. 
In  such  irregular  fights  and  in  emergency  situations,  the  legal  authority  to  engage  with  lethal 
force  is  more  often  conduct-based  and  thus  contingent  upon  an  enemy  demonstrating  a 
hostile  intent  or  engaging  in  a  hostile  act.  Given  the  higher  degree  of  difficulty  in  identifying 
targets  and  the  greater  distinction  concerns,  the  best  approach  may  be  to  avoid  using  LARs 

on 

under  these  circumstances.  Prudent  commanders  should  only  use  these  systems  in 
appropriate  situations  and  recognize  when  it  might  be  best  to  resort  instead  to  systems  with 
increased  human  oversight. 

Lastly,  LARs  should  be  required  to  have  some  version  of  a  human  override, 

O  1 

sometimes  referred  to  as  software  or  ethical  “brakes.”  The  systems  should  be  able  to  be 
shut  down  or  recalled  immediately  upon  a  commander’s  order.  Commanders  should  also 
establish  triggers  for  when  LARs  must  seek  human  guidance  before  engaging  a  target.  For 
instance,  when  a  LARs  system  identifies  expected  collateral  damage  greater  than  a 
predetermined  acceptable  limit,  it  could  be  forced  to  seek  guidance  from  the  command  before 
engaging  that  target.  Commanders  would  need  to  establish  protocols  and  support  structures 
to  facilitate  quick  decision  making  on  these  potential  targets.  In  these  circumstances,  human 
decision  makers  need  a  high  degree  of  clarity  about  what  situation  the  LARs  is  facing.  This 
oversight  would  not  be  effective  if  the  human  operator  were  merely  a  rubber  stamp  to 
approve  an  engagement.  With  prudent  additional  control  measures  such  as  these, 
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commanders  can  more  safely  use  LARs  on  the  battlefield  and  better  protect  themselves  and 
their  commands. 

V.  Conclusion. 

The  U.S.  will  likely  face  asymmetric  threats  in  military  campaigns  of  the  future. 
Whether  the  threat  is  the  substantial  jamming  and  cyber-attacks  capabilities  of  the  People’s 
Republic  of  China  or  the  legions  of  swarming  Iranian  patrol  boats,  LARs  may  provide  the 
perfect  solution  to  countering  these  threats.  LARs  have  the  unique  potential  to  operate  at  a 
tempo  faster  than  humans  can  possibly  achieve  and  to  lethally  strike  even  when 
communications  links  have  been  severed.  Autonomous  targeting  technology  will  likely 
proliferate  to  nations  and  groups  around  the  world.  To  prevent  being  surpassed  by  rivals,  the 
U.S.  should  fully  commit  itself  to  harnessing  the  potential  of  fully  autonomous  targeting. 

The  feared  legal  concerns  do  not  appear  to  be  an  impediment  to  the  development  or 
deployment  of  LARs.  Thus,  operational  commanders  should  take  the  lead  to  steer  this 
emerging  technology  into  becoming  a  true  force  multiplier  for  the  joint  force.  Operational 
commanders  who  establish  appropriate  control  measures  over  these  unmanned  systems  will 
ensure  their  LARs  are  effective,  safe,  and  legal  weapons  on  the  battlefield. 
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